-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add readme about dataset labeling #220
Conversation
Codecov Report
@@ Coverage Diff @@
## master #220 +/- ##
======================================
Coverage 20.8% 20.8%
======================================
Files 11 11
Lines 1216 1216
Branches 160 160
======================================
Hits 253 253
Misses 961 961
Partials 2 2 Continue to review full report at Codecov.
|
labeling_guide/ReadMe.md
Outdated
|
||
--- | ||
|
||
#### They look same. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They look the same
labeling_guide/ReadMe.md
Outdated
@@ -0,0 +1,56 @@ | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should move everything about labeling in this document, put it in the top-level directory calling it "LABELING.MD", and referencing it in README.md.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, that means we need to completely remove the labeling
part in README.md, and put a reference over there to "LABELING.md" which will be in the top-level directory, keeping images in labeling_guide
folder only (without the ReadMe.md
file which is there presently).
labeling_guide/ReadMe.md
Outdated
# Labeling Guidelines | ||
|
||
## Images marked as compatible - y | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's remove the newlines around the --- lines.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(if possible)
labeling_guide/ReadMe.md
Outdated
--- | ||
|
||
#### They are different. | ||
#### Mark the parts which are different logically with Green. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't mention the color here, we are going to mention it in the section where we explain how the labeling should be performed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
which are logically different
labeling_guide/ReadMe.md
Outdated
--- | ||
|
||
#### They look different. | ||
#### Mark the parts which are same logically with Yellow. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
which are logically the same
labeling_guide/ReadMe.md
Outdated
|
||
#### They look different. | ||
#### Mark the parts which are same logically with Yellow. | ||
>Different advertisements, different videos loaded, time-in-clock are marked |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's time-in-clock
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A screenshot which has present time displayed, it will be different in two screenshots as they are taken at different times but still compatible.
51b5856
to
88b3cae
Compare
88b3cae
to
6bc2305
Compare
LABELING.md
Outdated
#### They look different. | ||
#### Mark the parts which are logically the same. | ||
>Different advertisements, different videos loaded, time-in-clock, captchas are marked | ||
as different yet compatible. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also add the example which was in the readme about the news site.
Different videos loaded
shouldn't be there, the same video should be loaded in both browsers if the site is compatible. You can say different frames of a video loaded
.
Rename time-in-clock
as it is unclear.
I would say something like Since the screenshots are taken at different times in the two browsers, there are differences which are not incompatibilities but are actually due to the different timing. For example, a banner could be showing a different advertisement, a video could be in two different frames, a clock could be showing different time, a captcha could be showing different characters or images, two news could be different, and so on
.
LABELING.md
Outdated
#### Couples of images which are not compatible | ||
#### They are different. | ||
#### Mark the parts which are logically different. | ||
> Improper loading of images, missing text, different design, different languages are marked incompatible. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For example, improper loading ...
Add for example
because these are not the only cases.
9d87e70
to
06504a2
Compare
Thanks! I missed you made the updates to the PR... There are some conflicts though. |
The colors are wrong now, as we changed them in #221. |
@marco-c I will have to update the images in that case! |
5b58d52
to
090babf
Compare
@marco-c I have changed the images accordingly. |
Please provide your feedback on this pull request here. Privacy statement: We don't store any personal information such as your email address or name. We ask for GitHub authentication as an anonymous identifier to account for duplicate feedback entries and to see people specific preferences. |
No description provided.